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1 .  INTRODUCTION 


A  two-person,  zero-sun  stochastic  game  consists  of  a  (finite)  set 
S  of  states;  each  state  S  is  a  (finite)  matrix  game.  The  entries  of 
these  matrices  consist  of 

1)  a  payoff  (from  the  column-chooser,  B  ,  to  the  row-chooser, 
A  )  and 

2)  a  lottery  on  S  ,  determining  which  state  will  be  played 


next . 


Shapley  [1953]  introduced  this  concept,  studying  stochastic  games  which 
terminate  with  probability  1  after  finitely  many  stops;  equivalently, 
these  games  could  be  thought  of  as  infinite  in  duration,  but  with  a  non¬ 
zero  discount  rate.  In  this  case  the  min-max  theorem  is  straightforward 
(Shapley  [1953],  Monash  [1979,  1981]).  Gillette  [1957]  studied  stochastic 
games  with  zero  stop  probabilities,  establishing  the  min-max  theorem  in 
a  couple  of  special  cases.  In  these  cases,  the  optimal  strategies  are 
stationary  (i.e.,  dependent  only  upon  the  current  state,  rather  than  the 
history);  thus  the  game  "should"  go  into  a  Markov  chain.  The  payoff  can 
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be  defined  either  as  the  Cesaro  limit  lim  —  £  dN  or  the  Abel  limit 

N-*«“  1*1 

00 

lim  r  ][  d.  (1-r)*  ,  where  d.  *  the  payoff  on  the  i**1  plav,  since, 

r-K)  i-1  1  1 

with  best  play,  these  limits  exist  and  are  equal  (compare  Royden  [1963]). 

In  The  Big  Match,  Blackwell  and  Ferguson  [1968]  considered  a  more 
difficult  example.  Although  this  game  still  has  a  value,  it  cannot  be 
guaranteed  by  stationary  strategies;  furthermore,  no  strategy  is  better 
than  e-optimal.  Extending  these  methods,  Bewley  and  Kohlberg  [1976] 
showed  that  the  Cesar6  limit  of  the  values  of  the  N-stage  games  exists, 
and  equals  the  Abel  limit  of  the  values  of  the  r-discounted  games; 
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furthermore,  no  strategy  for  either  player  can  guarantee  an  average  pay¬ 
off  (in  any  sense)  better  than  this  number  v  .  Thus  v  is  the  only 
candidate  for  min-max  value.  Finally,  the  min-max  theorem  for  stochastic 
games  was  proved  by  Monash  [1979]  and  independently  by  Mertens  and  Neyman 
[1980].  This  paper  is  a  revision  of  Monash  [1979]. 


2.  DEFINITIONS 


A.*#c 


Without  loss  of  generality,  a  stochastic  game  can  be  described  /  -fy.  V, 

4V  - v 


by  finite  sets  S  ,  A  ,  B  ,  C  and  measurable  fun 


d  :  SxBxBxC-*  [-ft,  ft]  , 
s  :  SxAxBxC  +  S  ,  and 
q  :  [0,1]  -*■  C 


such  that: 


\  / 


1)  S  is  the  state  space; 

2)  Player  A  (resp.  B  )  chooses  a  move  from  his  choice  set 
A  (resp.  B  ); 

3)  s  ,  composed  with  q  ,  reproduces  the  lottery  in  each  entry 
of  each  state  matrix;  and 

4)  d  is  the  payoff  function. 


A  state  s*  €.  S  is  absorbing  if  4(s*,  a,  b,  c)  »  s*  for  all  a  ,  b  , 
c  and  d(s*,  a,  b,  c)  *  v(s*)  ,  a  constant.  S*  C  S  is  the  set  of 
absorbing  states  Sw  *  S  -  S*  . 

A  play  of  the  game  is  just  a  sequence  sQ,  a^,  bj,  c^,  s^,  a2, 
b2*  c2,  s2,  ....  where  s^  »  a^  b^,  c^  ,  for  all  i  ;  let 

^i  “  ^(®i-i»  *£»  c^)  •  the  payoff  on  the  iC^  turn.  Writing 

c^)  ®  T  ■  S*A*BxC  ,  we  denote  a  play  by 
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t  ■  (t^,  tj,  t^,  . ..)  ;  thus 

T*  -  S*AxBxC*SxA*BxC*S*... 

■  {all  possible  plays)  . 

The  subsequence  (t^,  tn)  is  denoted  by  -t(n)  ;  we  use  this  nota¬ 
tion  even  if  we  are  thinking  of  this  subsequence  as  belonging  to  many 
different  possible  plays. 

Strategies  for  A  will  always  be  denoted  by  o  ,  and  strategies 
for  B  by  t  .  These  strategies  will  always  be  of  the  form 

Prob(a  £  A  (resp.  b  £  B)  on  turn  k)  *  function(t^,  ...,  t^_^)  . 

Thus,  by  the  Kolmogorov  Extension  Theorem  (see  Kolmogorov  [1950]  or  Monash 
[1981]),  a  pair  (o,t)  determines  a  probability  measure  u(o,r)  on 

CO 

T  .  Unless  otherwise  noted,  all  expectations  below  are  with  respect 
to  this  measure.  Let 

T*  *  {(  6  T  :  si  €.  S*  for  some  i)  , 

and  T  the  complement.  In  the  next  section  we  write  P(*)  ■  p(T*)  . 

00 

Following  Bewley  and  Kohlberg  [1976]  or  Monash  [1981],  recall  that 

for  all  s  e  S  ,  for  all  r  6  (0,1)  ,  V_(r)  *»  the  value  of  the  r-discount 

8 

game,  starting  in  s  ,  satisfies 

V  (r)  -  val(exp(d(s,a,b,c)  + (1-r)  T  P(s)V-(r))  ,  (2.1s) 

8  c  s€S  8 

where  P(s)  ■  the  probability  that  4(s,a,b,c)  -  7  ,  and  val  is  the 
ordinary  min-max  value.  For  some  r  >  0  ,  all  the  Vs(r)  are  algebraic, 
as  are  the  optimal  strategies  in  the  games  (2.1s).  Thus,  on  (0,r)  , 
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Vg(r)  “  VJsJ-r"1  +  (  )r  n  +  ... 


-H 


V  ( 8 ) • U~n  +  (  )u"n+1  +  ...  , 


where  u  -  r1/n  .  Let  0  £  u  <_  r1/n  ;  on  (0,u)  ,  we  write  Wg(u)  -  VB(un)  , 


so 


that  lim  u\}  (u)  -  lim  rV  (r)  »  v  (s)  . 


u-K)  r-K) 

In  Sections  4  through  6,  we  assume  v  (s)  -  0  for  all  s  e  S  . 

00  CD 

In  that  case  we  have  lim  un-1W  (u)  <  ®  for  all  s  ;  thus,  writing 

+  8 

u-*0 

W(u)  *  max  |w  (u) j  ,  we  have  lim  un~Hf(u)  <  »  ,  also. 


seS 


u-*0 


3.  STATEMENT  OF  THEOREM 
Our  main  result  is 


Theorem  I:  For  any  starting  state  sQ  e  S  ,  for  any  e  >  0  ,  there 
exists  a  strategy  a  for  A  such  that,  for  any  strategy  t  for  B  , 


lim  inf  exp 
N-*» 


i  N 

5  £  di 
1«1  1 


>  v  (s) 


e  . 


Theorem  I  clearly  follows  from  the  following  two  propositions: 


Proposition  3.1:  Suppose,  for  all  s  4.  S*  ,  v^Cs)  «  0  .  Then  the  con¬ 
clusion  of  Theorem  I  holds. 


Proposition  3.2:  Proposition  3.1  — >  Theorem  I. 

In  this  section,  we  prove  Proposition  3.2;  the  remainder  of  the  paper 
Is  devoted  to  Proposition  3.1. 

The  proof  of  Proposition  3.2  depends  upon 


Lemma  3.3:  Let  G  be  a  stochastic  game,  with  state  set  S  .  Let  H 


be  another  stochastic  game,  identical  to  G  except  for  the  following 
modification:  Replace  a  single  state  x  £  S  by  an  absorbing  state  y 
such  that  v(y)  *  v^Cx)  .  Then,  for  all  s  e  S  , 

v-,H(s)  "  v-,G(s)  * 

where  ^(s)  (resp.  ^(s)  )  is  simply  v^s)  in  the  game  G 
(resp.  H  ). 

Proof:  Let  Vr  (r)  (resp.  Vw  (r)  )  be  V  (r)  in  the  game  (resp. 
w  p s  n,s  s 

H  ) .  Define 

\f(r)  =  r_1*voo(x)  -  Vg  x(r) 

V(r)  *  min  <VH  (r)  -  Vfl  (r))  . 
seS-{x}  H's  G‘s 

Then,  for  any  s  £  S  -  {x}  ,  (2.1s)  gives 

VH  _(r)  -  val(Exp(d(s,a,b,c))>  +  (1-r)  £  P(7)*VW—  (r) 

’S  C  a£S-{y)  h’S 

+  r"1* (l-r)P(y) *v(y)) 

>  val(Exp(d(s,a,b,c))  +  (l-r)_J  P(s)*Vp  -r-(r)) 
c  ses  b,s 

+  (l-r)  min  ((l-P)-V(r)  +  P*$(r)  , 

P6[0,l] 

where  P  corresponds  to  P(x  :  a,b,s)  , 

-  VG>s(r)  +  (l-r)((l-P*).V(r)  +  P**$(r)>  , 
for  some  P*  £  [0,1]  . 

X 

Picking  s  now  so  that 
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VH  £<r>  “  VG  “  v<r)  in  some  interval  10, r)  , 

V(r)  "  VH,s(r)  '  VG  s<r) 

>.  (1-r)  ( (1  -  P*)  *V(r)  +  P**$(r)) 

rV(r)  _>  (l-r)-P*-($(r)  -V(r))  . 

So  either  V(r)  0  ,  or  ^ (r)  -  V(r)  <  0  .  In  either  case. 


min  (v  H(s)  -  v  G(8)) 
s£S-{x}  * 


min  lim  (rVw  (r)  -  rVr  (r)) 

seS-{x}  .+  H’s  G’s 

r-K) 


*  lim  r  min  (V,.  (r)-Vr  (r)) 
^0+  s€S-{x}  H*S  G’S 


lim  rV(r) 
r->0+ 


>  0  or  >  lim  r^(r) 


r-K) 


So,  for  all  s  €  S-  {x}  ,  va  ^(s)  -  g(s)  _>  0  ;  that  is, 
v°°,H(s)  1  v»,G(s)  * 

But,  by  symmetry  (i.e.,  interchanging  the  names  A  and  B  ), 


Hence  ^(s)  »  vo  g(s)  ,  for  all  s  £  S  -  {x}  . 

Since  Lemma  3.3  is  clearly  true  for  state  x  ,  we  are  done. 

□ 
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Proof  of  Proposition  3.2; 

We  now  proceed  by  induction  on  | )  ,  the  number  of  non-absorbing 

states. 

| |  -0  .  Trivially  true. 

So  assume  for  |  |  —  1  ,  and  prove  for  | |  . 

To  every  state  s  ,  associate  a  number  a(s)  such  that 

1  N 

v  (s)  -  a(s)  ■  sup  inf  lim  inf  —  £  exp(d. )  . 

a  t  Ni*l  1 

By  the  Bewley-Kohlberg  result,  a(s)  _>  0  for  all  s  £  S  . 

Want  to  show:  a(s)  *  0  for  all  s  .  If  so,  done. 

So  suppose  otherwise. 

Definition:  We  will  call  a  strategy  o  ,  starting  in  state  s  , 
e-optlmal  (for  s  )  if 

1  N 

inf  lim  inf  £  exp(d.)  _>  v^Cs)  -  e  . 
t  ^  1*1 

Case  1:  There  exist  states  s^  ,  82  such  that  cKs^)  >  0(82)  >.  0  . 

Let  e  -  ^(aCs^  -  a(s2))  . 

Consider  the  modified  game  H  ,  where  &2  is  replaced  by  an  ab¬ 
sorbing  state  y  such  that  v(y)  *  v^^)  •  Then  H  ,  by  induction, 
has  an  e-optimal  strategy,  for  any  initial  state. 

Consider,  then,  the  following  strategy,  for  the  game  G  starting 
in  state  s^  :  Play  the  e-optimal  strategy  for  H  ,  until  "absorbed" 
in  "y"  ;  this  is  meaningful  because  G  and  H  are  identical  outside 
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of  state  S2  .  Once  in  S2  ,  play  an  (ais^)  +  e)-optimal  strategy,  which 
exists  by  the  definition  of  0(82)  .  Then  this  strategy  is  clearly 

(0(82)  + 2e)-optimal  for  . 


But 


0(82)  +  2e  =  -^(s^)  +  -|a(s2) 


<  a^) 


contradicting  the  definition  of  aCs^)  . 
Hence  the  only  possibility  is: 

Case  2:  There  exists  a  >0  such  that. 

Now,  let  v_  *=  min  v  (s)  . 

U  00 

ses 

ao 

Let  SnC  S  be  {s  e  S  :  v  (s) 

(J  —  OO  00  00 

Case  2a:  &  is  non-empty.  Then  let 

V  —  v 

0  1  0  1 

6  “  2(Vl+k)  -  2  • 


for  all  s  C  S  ,  a(s)  *  a  . 

OO 

*  vQ}  ;  let  S>  be  the  complement. 

*  min  (v  (s))  ;  v  >  v  .  Let 

seS  “  10 


By  repeated  applications  of  Lemma  3.3,  replace  the  states  in  S> 
by  absorbing  states  with  the  same  v  .  Then  the  states  in  Sn  still 

00  y 

have  value  Vq  .  Assuming  Proposition  3.1,  this  new  game  has  an  e-optimal 
strategy,  where 


Play  this  strategy  until  "absorbed,"  and  an  (a+e)- optimal  strategy  there¬ 
after  (unless  the  "absorption"  is  genuine).  Fixing  (any)  t  ,  we  have 


two  cases: 
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Case  1: 


since 

Case  2 

Let  Y 


Hence 


since 


v0  +  vl 


Expected  value  if  "absorbed"  _>  - -  or  P(*) 


i 

lim  inf  rr  £  exp(d.)  _>  P(*) 
N+*  i-1 


v0  +  vl 

-V-i  -  (a+e) 


+  (1 


Vq  -  a  +  P(*) 


V1‘V0 


•  -  e 


_>  Vq  -  a  +  min 


vl"v0  7a 


4  »  8 


r  was  arbitrary,  this  contradicts  the  definition  of 


V0  +  Vl 


Expected  value  if  "absorbed"  <  - ^ -  and  P(*)  > 


prob (genuine  absorption) 

P(*)  •  lhen 


vo+vi 


2 —  >  Expected  value  if  "absorbed" 
>  y(-&)  +  (I-yJ'Vj  ;  i.e., 


v,  - 


V1"V0 


1  2 


>  v  -  Y^+M) 


(vi  -  V  „ 
Y  >  m  6 


lim  inf  £  exp(d.)  vn  -  e  -  P(*)  (1-y)  (a+e) 
N-*«  Ni-l 

2^  Vq  -  e  -  (l-$)  (a+e) 

>  Vq  -  a  +  (8a  -  2e)  ; 


„  ea 

e  -T  • 


0  .  Then 

-P(*))(vQ-  e) 
(l-P(*))(a-e) 

a  . 

•  0  . 


this  is  a  contradiction. 
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Case  2b:  fa  =  4>  . 

Deducting  Vq  from  all  payoffs,  this  is  exactly  the  case  of  Pro¬ 
position  3.1.  Hence  there  exists  an  -j- optimal  strategy,  for  our  final 
contradiction. 

So  a(s)  =  0  for  all  s  e  S*  . 

But  this  is  exactly  what  we  wanted  to  prove. 

□ 


4.  PRELIMINARY  COMPUTATIONS 


For  the  rest  of  this  paper,  we  will  assume  v^s)  =  0  for  all 

s  £  Sw  .  We  will  always  choose  A's  strategy  a  to  be  Prob(a)  *  f(a,s,u) 

“  the  optimal  (stationary)  strategy  in  the  un-discount  game,  for  some  u  , 
in  the  current  state  s  .  Without  loss  of  generality  (see  Monash  [1979) 
or  (1981)),  B’s  strategy  t  is  pure:  b^  *=  function (£(k-l) )  . 

a. 

Let  us  now  focus  on  one  move  of  the  game.  Fix  s  €  ,  u  6  (0,u)  , 

and  b  €  B  ,  with  A  playing  strategy  {f(a,s,u)J  .  Let  P*(u) 

*  Prob(4(s,a,b,c)  £  S*)  ,  given  the  probability  distributions  f(a,s,u) 
on  A  and  q  on  C  .  In  Sections  5  and  6,  if  a  play  t  is  understood 
along  with  a  sequence  of  u's  ,  we  will  let 


P*(i) 


P*(u)  on  the  iC^  turn,  if 
0 


if 

s .  , 

6 

s 

i-1 

00 

if 

si-l 

e 

s*  . 

Meanwhile,  let  s  *  4(s,a,b,c)  . 

We  distinguish  three  cases: 

1)  P*(u)  H  0  on  (0,u)  ; 

2)  Not  1,  and 
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lim  expfv^Cs)  :  s  €  S*)  »  0  ; 
u-*>+ 

3)  Not  1  or  2. 

We  further  distinguish  between: 

A.  Either  Case  1,  or  order(P^(u) )  >  n  ; 

B.  Not  Case  1,  and  order(p*(u))  _<  n-1  . 

Observe  that  P*(u)  is  a  rational  function  of  u  ,  and  thus  has 
finitely  many  zeroes;  without  loss  of  generality,  none  of  them  occur  on 
(0,u)  .  Define  S(u)  as  follows  (where  we  suppress  the  dependence  upon 
s  and  b  ) : 

If  Case  A,  then 

5(u)  *  -exptv^s)  :  s  6  S*)*PA(u)*u  n  ; 
if  Case  B,  then 

6(u)  =  -exp(vw(s)  :  s  £  S*) *P*(u)  +  (1  -  (l-u11) (l-P*(u))Wg(u) *u  n  . 

The  point  of  this  definition  may  be  found  in  the  following  propositions 
(where  we  write  exp(d  :  S^)  for  exp(d  :  s  £  S^)  ,  and  so  forth): 

Proposition  4.1:  exp(d  :  Sw)  (1  -  P*(u))  >_  6(u)  -  exp(W^-- Wg(u)  :  S^)  (1  -  P*(u) ) 

+  n(u)  ,  for  u  £  (0,u)  ,  where  lim  n(u)  ■  0  . 

u-*0+ 


and 
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Proposition  4.2: 

1.  If  Case  1  (above),  then  6(u)  -  0  and  P*(u) ‘expC^ 

2.  If  Case  2,  then 

lexpCv^s)  :  S*)  |  <  o(u°) 
and 

6(u) -un 
P*(u) 

3.  If  Case  3, 

-P*(u)  •exp(voo(¥)  : S*)  •  u”n 

600  -  1  +  o  (u  )  . 

Fron  Equation  (2.1s).  we  have 


<  0(u°)  . 


ws(u>  i  (1-  P*(u))«(exp(d  :  sj  +  (1  -  un) ‘exp(W^(u)  : 
+  P*(u)  •exp(voo(7)  :  S*)u_n  . 


Proof  of  Proposition  4.1: 

Rearranging  (4.3),  we  have 

(1 ~  P*(u))exp(d  :  S  )  >  (1  -  (1  -  P  (u) ) (1  -  un) )W  (u) 

s 

-  (1  -  P*(u))  (1  -  un)«xp(W^-(u)  - 

-  P*(u)exp(vco(‘s)  :  S*) 

If  Case  A  holds,  then  (4.4)  -  6(u)  -  (1  -  P*(u))exp(W^-(u)  - W( 

+(P*(u)  +un  -unPJk(u))Wg(u)  +  ur 
•  exp(W^(u)  -Wg(u)  :  SJ  . 


s)  :  S*)  ■ 


*.» 

(4.3) 


W  (u)  :  S  ) 
8  ® 

(4.4) 
,(«>  :  SJ 
(1-P*(u)) 

(4.5) 
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If  Case  B.  holds,  then  (4.4)  equals 

6(u)  -  (1  -  P*(u))  *exp(VH-(u)  -  Wg(u)  :  Sa) 

+  un(l-P*(u))*exp(W-(u) -W  (u)  :  S  )  .  (4.6) 

Let  P  >  0  be  such  that  |PA(u) |  ±  Pun  whenever  Case  A  holds.  Writing 
n(u)  “  -(P+4)uI1W(u)  , 
and  observing  that 

(4.5)  _>  6(u)  -  (1  -  P*(u))  *exp(W^-- Wg(u)  :  S^)  +  n(u)  , 

(4.6)  5(u)  ~  (1  -  P*(u))  *exp(W^-- Wg(u)  :  S^)  +  n(u) 

and  lim  n(u)  =0  , 


we  are  done. 

□ 

Proof  of  Proposition  4.2: 

1.  Suppose  Case  1  holds:  P*(u)  =  0  .  Then  so  does  Case  A,  and 

6(u)  *  -exp(voo(s)  :  s  £  S*)  *P*(u)  *u”n 
*  0  for  all  u  , 


and  so  done. 

2.  Suppose,  then,  Case  2  holds.  Since  exp(v<B('s)  :  S*)  is  a  power  series 
in  u  ,  with  limit  0  as  u  ■*  0  ,  it  is  Indeed  o(u^)  .  Now,  if  Case  A, 


then 
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6(u) »u 
P*(u) 


|-exp(v  (s)  :  S*} 


<  o( uu)  ; 


while,  if  Case  B,  Chen 


v  n  P*(u)*W  (u)‘u 

“  -exp(Va)(s)  :  S*)  + - - +  higher  order  terms 

<  o(u^)  +  lii1^  (u)  |  +  higher  order  terms 


<  °(u  )  • 


3.  Suppose  Case  3  holds.  If  Case  A,  then 


-P.(u)-exp(v  (s)  :  S*)  u 


=  1  ,  by  definition. 


So  suppose  Case  B:  order  P*(u)  <_ n-1  .  It  is  clearly  enough  to  check 


(P*(u)+un-u\(u))«s(u)  0 

- — - —  <  0(u  )  . 

-P^(u)  *exp(  v^s)  :  S*)u” 


Then  order  (P*(u)  + un  -  unP*(u) )  ■  order  (P*(u)) 


order  (exp(v  (s)  :  S*))  ■  0  , 


and  so  the  order  of  the  left-hand-side  is 


>  order  (P*(u))  +  order(W  (u))  -  order  (P.(u))  -  0  -  order  (u-n) 


order  (u1^  (u)) 
s 


Hence  done. 


5.  THE  ABSORBING  CASE 

Recall  that  a  fixed  strategy  pair  (o,i)  induces  a  probability 
measure  u(o,t)  on  T°°  ,  the  space  of  all  possible  plays.  If  8q  £  S* 
Proposition  3.1  is  trivial;  thus  it  follows  immediately  from 

Proposition  5.1:  For  any  starting  state  s  £  ,  for  any  e  >  0  ,  the 

exists  a  strategy  a  for  A  such  that 

N 

inf  lim  inf  /  —  £  d.dp(o,T>  >  -(6&+3)e  . 

t  N-"»  T*  i-1 

Proof  of  Proposition  5.1: 

As  remarked  earlier,  the  strategy  o  will  be  the  form 
Prob(a)  *  f(a,s,u)  ,  the  optimal  strategy  in  the  un-discount  game,  for 
u  cleverly  chosen.  Specifically,  writing  u^  for  the  u  prevailing 
on  the  N+lst  move,  we  set  u^  *  Uq(1--^e)V^  ,  for  u^  sufficiently 
small  and  v(N)  a  non-negative  integer  depending  upon  the  history  of 
the  first  N-l  moves. 

Write  q  ■  1  -  -je  .  Recalling  Proposition  4.2,  choose  R  >  0  and 

'v-  0 

u  sufficiently  small  so  that  each  0(u  )  is  <  Ru  .  Assume  e  <  1  . 

% 

Then  u^  €  (0,u)  c  (0,1)  must  satisfy  the  following  four  conditions: 


1. 

For  every  u  e 

(0,u0J,n(0)  >  -i 

2. 

For  every  u  e 

(0,u],W(u)  <  ■|-u 

3. 

%<E  jn 

4. 

<1+E)  TV71 

1  -  q 

<  E  . 

% 

,  we  first  define  a  set  of  benchmarks  m  on  1,  0,  1,  2 


To  define  v(N) 
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m(-l) 

m(0) 

«(i) 


m(l-l)  +  (u^1"1)  2 


for  i 


1.  2,  3, 


Next,  define  sequences  mQ,  m^  m2,  ...  and  JC  •  (iQ-0,  l  t  i  ...)  , 
&  increasing, in  conjunction  with  the  sequences  uQ,  Ul,  u^  ...  and 
v(0),  v(l) ,  v(2) ,  ...  by: 


1) 

m0  -  0 

2) 

o 

1 

/'“'S 

o 

3) 

v(N) 

“n'V 

for  N  -  1,  2,  3,  ... 

4) 

If  Vi  +  4n 

(uN-lJ  >  ®Cv(N-D+l)  ,  then  V(N)  -  v(N-l) +1 

and  N  £«C  ; 

if  “jj.!  +  6n^“n-1^  <  ®(V(N-1)-1)  .  then 

v(N)  »u(N-l)-l  and  N  eX  ;  otherwise  v(N)  »  v(N-l) 
and  N  t  £  . 

5)  If  N  t  X  ,  then  -  m^  +  ^(“n-I*  . 

6)  If  N  -  €  t  ,  then  ^  -  V*  +  {N(Vl)  +  Ws  <Vl> 

"V  <Vl>  *  ^ 

i 


Fix  o  as  above,  and  any  (pure)  t  .  Proposition  5.1  follows 
instantly  (by  redefining  t  )  from: 

,  N 

Proposition  5.2:  lim  /  ~  £  d.dp  _>  -e 

N-+®  T*  Ni-1  1 

and 


1 

Proposition  5.3:  lim  inf  /  ±  j>  d.du  >  -c  . 

N-x»  T  i-1  1 
00 

We  now  prove  Proposition  5.2,  deferring  Proposition  5.3  to  the  next 


section. 
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Proof  of  Proposition  5.2: 

Let  Tfc  -  U  -  (tlt  t2,  ...)  €  T*  :  4(tfc)  S*  but  4(tk-1)  4  S*) }  ; 
thus  T*  ■  U  T2  u  T3  u  . . .  • 

,  N 

So  li «  /  41  d  dv 

N-*»  T*  i-1 


-  lim  l 
N-*»  k-1 


i  N 

f  Z*, 

wi-l  J 


dy 


-  J  lim 
k-1  N-k” 


/  |  I  ^dy 


1-1 


by  the  Lebesgue  Dominated  Convergence  Theorem  (Royden  [1963]), 


-  I 

k-1 

00 

-  I 

k-1 


/ 


/ 


lim  i  l  d  dy 
N-~  i-1  1 

PA(k)  •exp(va>(st)  :  t  €■  Tk)dy  • 


T 


(5.4) 


The  following  is  a  special  case  of  Proposition  4.1  of  Monash  [1981] 
(identifying  Z*  -  4*‘1(S*)  for  all  i  ). 

'V/ 

Proposition  5.5:  There  exists  a  probability  measure  y  on  T  such 
that,  for  all  N  ,  for  all  fN  :  TN_1  -*■  F  such  that  4(tN_x)  €  S* 
implies  fN(t1,  •••»  t^)  *  0  * 

N-! 

/  fN(*(N-l))du(H)  -  /  fN(-£(N-l)).  n  (l-P*(i))dy(-t) 

•  •  i-1 

T  T 
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Now,  assume  temporarily. 


Proposition  5.6;  For  all  N  ,  for  all  t  , 


Let 


N  k-1 

l  (P*(k)-exp(v<B(sk)  :t€  Tfc)  •  n  (1-P*(i)))  >  -c 


fk(*(k-l>) 


P*(k)  -expCv^Cs)  :t  £  Tk>  if  4(tkl>  ^  s* 
0  if  4(tk-1)  6  S* 


Then  fk  satisfies  the  hypothesis  of  Proposition  5.5.  Thus,  for  all  N  , 


N 

I  P*(k)*exp(vao(sk)  :t£T) 
k-1 

N 

-  I  /  fk(*(k-l))dp 
k-1  T“ 

N  k-1 

-  I  /  f,(*(k-l))-  n  (1  -  P*(i))dp 

k-1  »  i-1 


*  /  1  P*(k)  ’expCv^Csj^)  :  t  6  Tk)  •  II  (l~P*(i))du 

T®k*l  i“l 


>  /  (— e) dy  ,  by  Proposition  5.6, 
00 
T 

-  — £  ; 


as  these  are  the  partial  sums  of  equation  (5.4),  this  establishes  Propo¬ 
sition  5.1. 


So  we  pass  to  the 
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Proof  of  Proposition  5.6: 

Fix  t  and  N  .  Recalling  Proposition  4.2,  we  make  the  simplify¬ 
ing  assumption  that  Cases  1  or  3  hold  everywhere  (for  fullest  detail  see 
Monash  [1979]);  thus,  for  k  ■  1,  .  N  , 


either  ^(u^)  “  P*(k)  ‘expCv^Csj^)  :  Tfc)  -  0 
-P*(k)  •exp(voo(sk) 


or 


6<\-l) 
C  (1-e,  1+e) 


e  (1-Ruk  l,  1  +  Ru^) 
(5.8) 


k-1 


Writing  F,  *  P*(k)  exp(voo(s.)  :T,)*  II  (1-P*(i))  »  our  task  is  to  bound 


N 


i-1 


IN 

£  F(k)  below.  We  spread  out  this  sum  as  the  integral  of  a  step  func- 
k-1 

tion  by  defining  A(z)  on  [0,N)  :  A(z)  *  F([z]+1)  ,  where  [z]  is 

Fk 


N  N 

the  usual  greatest  Integer  function.  Thus  /  A(z)dz  *  £  F,  . 

0  k-1 


First ,  observe  that ,  for  6^  , 


Ws  (ui  >  '  Ws  (ut  > 

_ V*  J_ 


V  “"I 


e  (l-e,  1+e)  , 


f 


6k(ui  5 

k-yik  j 


f 


6k(ut  ) 

k-yik  j 


since 


__  -n+y 

|wfi  (u»  >  -  K  )li  2W<\  )  <  f  u. 

8*j  s*j+1  *j  V  2 


i  i 

1+1  -n+4 

while  |  l  d.  (u.  )  |  %  u.  ,  by  definition. 

k-y1  j  i 


Thus  we  can  define  a  function  m(z)  on  [0,N]  such  that 
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We  now  want  a  finite,  increasing  sequence  J  -  {0*Jq,  j^,  j 2 ,  ..-,N}  , 
containing  every  integer  0,  1,  N  ;  J  should  be  partitioned  into 

five  sets,  with  the  following  properties: 

1)  For  j  £  ,  m  is  constant  on  [ j ^ ,  j^+^]  . 

2)  For  J2  or  ,  m  is  increasing  on  [ji,  j^+^]  . 

3)  For  j  £  or  Jj  ,  m  is  decreasing  on  [ j ^ ,  ji+^]  . 

4)  There  exists  a  bijection  $  :  J2  -*■  such  that  if  4>(j^)  *  » 

1)  i  <  h 

2)  tn(jh)  - 

3)  “^h+l5  *  m(ji)  * 

5)  For  any  jh,  ^  6  J 3  with  h  <  i  ,  m(jh)  <  . 

Let  Ji  "  .  ^  lV  W  ’  i  -  1,  2,  3,  4,  5  . 

Jh  Ji 

6)  The  sets  J  cover  10,N)  . 

Lemma  5.7.  There  exists  such  a  sequence  J  . 


Proof  (See  Figure  1): 

We  will  prove  this  lemma  by  induction  on  the  number  H  of  maxima 
attained  by  m  on  the  interval  tO,N]  (this  number  is  <  N+l  ,  and 
hence  finite) . 

Clearly  it  will  be  enough  to  construct  the  sets  J^,  . 

H  -  1  :  Then  m  is  either  monotonic  non-increasing  or  monotonic 
non-decreasing.  In  the  first  case,  let  be  the  set  on  which  m  is 
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strictly  decreasing,  and  the  balance;  in  the  second  let  be  the 

set  where  m  is  strictly  increasing,  and  the  balance. 


Inductive  step:  Assume  true  for  H  . 

So  suppose  the  maxima  occur  at  y^,  ...,  y  »  and  the  minima 
at  (Xq) ,  Xj,  x2,  ....  Xjj,  ,  so  that  xjL_1  <  <  x±  »  for 

i  ■  1,  ...»  H+l  (  Xq  or  Xy+^  may  not  exist).  Apply  induction  to 
[0,  Xjj]  (recall  that  m  changes  direction  only  at  integral  arguments), 
and  construct  a  tentative  J^,  ...,  .  Then,  for  every  point  j  in 

[3^,  yH+1l  »  either  there  exists  i  €  C  [0,  Xjj)  such  that  m(i)  m(j) 
or  else  not.  In  the  first  case,  put  j  into  and  move  i  into  ; 

in  the  second,  put  j  into  .  Finally,  put  IyH+j*  N)  into  . 

It  is  clear  that  this  is  the  desired  partition. 

□ 

Our  result  is  now  clear  (again  redefining  e  )  from  the  following 
four  lemmas: 


Lemma  5.10a: 


Lerana  5.10b: 


Lemma  5.10c: 


Lemma  5.10d: 


/  A(z)dz  ■  0  . 


/  A(z)dz  >  0  . 


/  A(z)dz  >  -e  . 


/  >  -6Me  . 

J2  U 


Lemma  5.10a  is  obvious,  since  A(z)  *  0  on  by  construction. 
Similarly,  A(z)  is  increasing  on  ,  and  so  Lemma  5.10b  holds. 
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Proof  of  Lemma  5.10c: 


A(z)dz  -  l  A([j  ]).(j  - 

j3  ifJ 


[jj 

>-(1+e)2  l  u"  ,(«(!,) n  (1  -  P*(D)  , 
j1eJ3  uiJ  1+1  1  i-1 

by  (5.8)  and  (5.9), 


>  -(1+e) 2 


(1+e)  l  (u_q*)  (m(l+l)  -  m(l)) 
1-0 


-  -(1+e)3  l  (u0q*) 
1*0 


1/2 


-  -(1+e)3 


1/2 
0 _ 

Q1/2 


>  -e  . 


Proof  of  5.10d:  Let  y  : 


A(z)dz 


J2UJ4 


J  ^  J  2  be  <}>  . 

31J2 


2  n  Uj  1 

>-  l  (l+e)^(m(j  )-m(1  .)).u"  .  jf(l-P*(i)) 

3fJ2  11  -i  [ji]  l-l 

IJ±1 

-  I  (l-e)2(m(j  )-m(j  )).u"  .  n(l-P*(i)) 

Jj€J4  11  1  lJiJ  i-1 

UiJ 

-  JiJ4(<1'c)""j1r<l+E,2ui^i1)])<"(3i+i)  £ (1- 

(5.11) 


by  the  defining  properties  of  $  . 


P*(i» 
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Now>  ufr ( j±) ]  ”  q  uIj1]  '  vhere  x  "  "lf  °*  1  (to  8ee  thl8» 
observe  that  if  m(i)  <  ^  ^  m(i+l)  ,  v(N)  must  -  i  or  i+1  ) .  Thus 
(5.11) 

1 3*1 

>.  ( (1-e) 2  -  q  n(l+e)2).  £  u?.  ,  (m(j  .  .  )  -  m(j  . )  .  n  (1-P*(i)) 

j^J4  lJiJ  1+1  1  i-1 

“  2  It— 1 

>  -5e  l  M(l+e)%(k)  n  (1-P*(i)) 


k=l 


i-1 


by  the  properties  of  m  and  the  fact  that  jv^s*)  |  _<  ft  for  all  s*  fc  S* 

>  -6fte  . 


This  completes  the  proof  of  Lemma  5.10d,  hence  of  Proposition  5.6, 
and  hence  of  Proposition  5.2. 


6.  THE  NON-ABSORBING  CASE 
We  now  prove 

Proposition  5.3: 

1  N 

lim  inf  /  -  J  d  dp  >  -e  . 

T-*»  T  "i«l 
00 

Proof: 

Lemma  6.1: 

Let  (eijc^k^co  converge  uniformly. 
1  N 

Let  E  -  lim  inf  ±  £  (lim  e..  ) 

N-**>  i-1  k-*»  lk 

1  ? 

Then  E  -  lim  inf  rr  £  e..  . 

N-*“  Nj-1 
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Proof:  Easy. 


□ 


Identifying  e  as  / 


Jk  «  j  d.dp  ,  we  observe  that 

u  • UTk)c 


I  /  Adu-J 

T 


(T^J...  UTk) 


d.dp|  _<  ft  Y  p(T,  )  -►  0  as  k  -*• 
c  k+1  k 


Thus  Lemma  6.1  gives 


iim  inf  i  [  j  d  dp 
Hh“  i-l  T 


lim  inf  i  l  j 

_  41  J  1 


N-x” 


i=l  (T.U 


.  U  T±)c 


didy 


Let  oj^(u)  =  ws  (u)  ~  (u)  .  Then,  by  Proposition  4.1, 


ai-l 


,rdidy-/  (6i(ui-l)  ““iK  ,)+n(u.  .))du 

(T1U...UT1)C  (T1U-..UT1)c  1  i'1  1  1-1  ^ 


l! 


(Tj  U  ...  UT^c 


<6i(ui_l)  -  «i(ui_i))du  -  e*u(CTiu ... u  Tc)c)  . 


Of  course,  also. 


(Tx  U...  UTj)® 


d±dy  >  -^((Tj^U...  UTc)C)  . 


Thus,  setting 


fk(*(k-l))  .  \ 


max ( -ft,  6k(uk_1)  -  ^(u^))  if  4(tk_x)  *  S* 
0  otherwise. 


we  see  that 


Z3 
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uTt)C 


djdy  >_  / 


(T1u...  UTi)‘ 


f<(*(i-l))dp  -  e 


-  /  f1(*(i-l))dii  -  c 


Hence,  to  establish  Proposition  5.3,  it  is 


Enough  to  show: 


1  N 

liin  inf  i  l  J  fk(i(k-l))dy  >  0 


N-*”  k-l  « 


(6.2) 


Applying  Proposition  5.5,  for  each  N  , 


r  v  .  S  k-l 

/  1  fkWk-i))dv  -  /  l  f  •  n  (l-p.(D)du  . 

T-  k-l  K  «  k-l  k  i-i 


i  _  ,  .  N  k-l 

iim  inf  —  £  /  f.dp  -  lim  inf  /  A  £  f  .  n  (1-P*(i))dp 
N-«o  K-l  «  N-*»  »  \-l  k  i-1 


.  N  k-l 

>/  lim  inf  A  l  f  n  (1-P*(i))du  , 


®  N-h»  "k-l  i-1 


by  Fatou's  Lemma  (Royden  [1963]). 
So,  if  we  establish 


Lemma  6.3:  For  all  U  T*  , 


,  N  k-l 

lim  inf  -  l  f.(t(k-l)).  n  (1-P*(i))  >  0  , 
N-*»  Nk«l  *  i-1 


we  are  done. 


rf  ' 


’  VW?.|'TMii»i»{ir  KT 
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Proof :  Suppose  we  know 


.  ii  aw 

lim  inf  i  J  f.  (-t(k-l))*  n  (1-P*(i))  >  0  . 
N-*»  k-1  i-1 


Then  either  there  exists  N  such  that  k  >  N  implies  that 
1  -  P*00  >  ,  in  which  case  we  are  done  immediately,  or  else 


N  k-1 

lim  inf  ^  l  f  (-C(k-l))-  n  (1-P*(i)) 
N-*»  k-1  i-1 


_>  lim  inf 
N-*“ 


N  k-1 

-Si  n  (1-P*(i)) 
i-i 


o  . 


But  we  can  in  fact  show  the  stronger 


.  N  k 

lim  inf  ±  J  f'(t(k-l))«  n  (1  -  P(abs  :  t,  i-1))  _>  0  , 
N-*®  k-1  i-1 


(6.3' 


where 


fk  "  6k  -  “k  ±  fk  • 


Letting  P^  =  P*(i)  ,  for  all  i  ,  we  have 


N  k 

l  n  (i  -  p.) 

k-1  i-1 


N  N 

I  f/.  n  (l-p  ) 

k-i  i-l 


+  l  (  l  f •  n  (l-p  ).p  ) 

j-i  k-i  Ki-i  1  3 


(6.4) 


Lemma  6.5;  There  exists  a  number  fig  such  that  for  all  t  ,  for  all 

N0 

N  ,  for  all  N_  ,  J  f'  <  0  implies  that 

k-1  K 


N 


I  fi*  n  (i-P*(D)  >  -on  . 

1  .Li  ^  i  — ' M  l  *1  ^ 


k-NQ+l  i-N0+l 
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(In  particular,  this  conclusion  holds  when  Nq 


N0 

Proof:  y  f,’  <0  implies  that  u*.  »  u«  or 

-  k  _  N0  u 

N 


y  f£  ■  -M  -  2W(u^)  ;  assume 


k-N0-U 


N  N 

M  ^  0(else  £  f' •  II  (1  -P.)  _> 
k-N  +1  K  i-NQ+l 


Then 


I  «k<  I  (6-w)+2W(u) 

t  in  ^  i _ i  -i  IV  X 


k»N0+l  "  k-NQ+l 


-M  -  2W(ux)  +  2W(ux) 


-M  . 


Recalling  Proposition  4.2,  and  noting  that 


|v  (s)  |  _<  M  for  all  s  €  S  , 


(6.6)  implies  that 


hence 


k’V1 


i  ^-hu; 


Mu" 

ST' 


N-l  N-l 

l  (1  -  P*(k)  <  n  (1-P*(k)) 


k-NQ+i 


k"N0+1 


Mu' 


< 


0  .) 

.  Write 

-2W(Ul))  . 

(6.6) 
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by  a  well-known  Inequality  (which  can  be  derived  immediately  from  the 
observation  ln(l-P)  <  -P  for  0  <  P  <  1  ).  Thus 

N  N 

I  fu  n  (i 

k«NQ+l  K  i-NQ+l 

So  if  we  set  fig  =  3&u^n  ,  we  are  done. 


-P1)  >  (-M-2W(Ul))e 


Muj 

~W 


2ft  1  ~T,  . 

- -  2W(u  ) 

un  e  1 


Returning  now  to  the  proof  of  (6. 3'), we  distinguish  two  cases: 


Case  1:  £  P  <  °°  . 

k»l  k 


Then 


.  N  k 

lim  inf  ±  l  fk-  n  (1-P  ) 
N-*»  k-1  i«l 


.  il»  +  Z  <-  Wl’  ’ 

N-x»  j«l  J 


by  (6.4)  and  Lemma  6.5, 


nn  » 

>  lim  inf  -  -f  •  1  +  l  p 
N-*»  "J  k-2  K 

-  0  . 


OD 

Case  2:  J  P.  -  •  . 

k-1  K 
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Case_2a:  There  exists  Nj  such  that  N  >  implies 


Then 


k  _ 


hence 


N  k 


l  V  n  (i-p.) 

c-1  K  i-l  1 


n  jj  i  j 

n  (1"pi>  +  l  <  Z  V  n  U-PJ •?,.,) 

“1  i-l  1  j-1  k-l  k  i-i  1  J+1 

N-i  i  k 

+  I  (  I  f.  *  n  (i  -p  ). 

j-Nx+i  k-l  *  i-l  1 


•p  ) 

j+r 


a. 

—  “no  +  j^”nO^’Pj+l  +  Pos^tive  terms; 


,  N  k 

lim  inf  -  l  f  .  n  (1  -P.) 
N-x»  k-l  K  i»i  1 


>  lim  inf  SSSgSSl  , 
N-*->  N 


Case__2b:  There  exists  no  such  Nj.  .  Then  for  arbitrary  e*  >  0 
exists  N*  such  that 


N* 

I  (1  -  Pj)  <  e*  and 
i-l  1 


N* 


<  0  . 


Let  N  >  N*  . 
Then 


,  there 


.■  MZ 
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N  k 

X £k-.n.(1-pi> 


k-1  "  i-1 


N*  k 


-  [f-n(i-p)+  l  f.  n  (i-p  ).  n  (i-p.) 

k-l  1-1  k-N*+l  k-N*+l  1  i-1  l 


>  -o  _  y  o  .p 

-  0  jf1  0  *j+l 


N-l 

-  e*  -  n©  -  £  %*Pi+l 

U  j-N*+l  U 


hence 


by  Lemma  6.5  and  a  slight  extension  of  (6.4), 


_>  -constant  -  Ne*0q  ; 


,  N  k 

11m  inf  ±  l  f  •  n  (1-P.)  2 

N-k»  "k-l  K  i-l  1  u 


But  e*  was  arbitrary: 


hence 


1  N  k 

lim  inf  i  l  f  •  n  (1-P.)  >  0 
N-kc  "k-l  i-1  1 


This  completes  the  proof  of  equation  (6.3'),  hence  of  Lemma  6.3,  hence 
of  Proposition  5.3,  hence  of  Proposition  5.1,  hence  of  Proposition  3.1, 


and  hence  of  Theorem  I. 


Q.E.D.  □ 
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